perf: optimize binary scanning with streaming and concurrent chunk processing - #96
Merged
Merged
Conversation
Previously flipFuses and getCurrentFuseWire loaded the entire Electron binary into memory, scanned it, and (for flipFuses) wrote the whole file back to disk even though only a handful of bytes change. This switches to a chunked scan that holds ~8 MB in memory regardless of binary size, runs two workers concurrently so indexOf overlaps with disk reads, and uses positional writes to touch only the modified fuse bytes. On synthetic binaries this yields roughly: flipFuses 150 MB: ~350ms -> ~35ms flipFuses 300 MB: ~830ms -> ~58ms getCurrentFuseWire 300 MB: ~435ms -> ~28ms
Only 1 sentinel (single-arch) or 2 sentinels (universal macOS) are valid. Finding more indicates a corrupted or unsupported binary, so fail loudly rather than silently patching an unknown layout.
MarshallOfSound
changed the title
Restores the atomicity guarantee of the original implementation: if validation fails for any sentinel (version mismatch, strictlyRequireAllFuses), the binary is left completely unmodified. Previously, with per-sentinel writes inside the validation loop, a universal binary could end up with one arch slice modified and the other untouched if the second slice failed validation.
erickzhao
approved these changes
Mar 26, 2026
|
🎉 This PR is included in version 2.1.1 🎉 The release is available on: Your semantic-release bot 📦🚀 |
MarshallOfSound
added a commit
to electron/packager
that referenced
this pull request
Mar 31, 2026
Previously setIntegrityDigest loaded the entire Electron Framework binary (150-500 MB) into memory, scanned it, modified 34 bytes, and wrote the whole thing back. Now it scans in 4 MB chunks with two concurrent workers (I/O overlaps Buffer.indexOf CPU) and patches only the 34-byte digest slot(s) via handle.write at the found position(s). Each chunk overreads sentinel.length-1 bytes so a sentinel straddling a boundary is still detected. Peak memory drops from ~binary-size to ~8 MB. Same approach as electron/fuses#96, which benched 10-15x faster on 150-300 MB binaries. Adds two synthetic-binary tests: one plants the sentinel across a 4 MB boundary (verified load-bearing via mutation), one plants a sentinel in each of two chunks to cover the universal-binary multi-write path. Both skip packager() and run in ~10ms.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes
flipFusesandgetCurrentFuseWireroughly 10–15× faster on typical Electron binaries, while capping peak memory at ~8 MB regardless of binary size.Previously these functions loaded the entire Electron binary (100–300 MB) into a single buffer, scanned it, and — in the case of
flipFuses— wrote the whole file back to disk even though only a handful of bytes change.Benchmark results
Measured on synthetic binaries with embedded fuse wires (median of multiple runs):
flipFusesflipFusesflipFusesgetCurrentFuseWiregetCurrentFuseWireMemory: peak usage drops from ~binary-size to a fixed ~8 MB.
How
findSentinels()reads the file in 4 MB chunks instead of all at once. Each chunk overreadsSENTINEL.length - 1bytes into its neighbour so a sentinel straddling a boundary is still detected.Buffer.indexOfon one chunk runs while the next chunk is being read from disk. Benchmarking showed concurrency=2 is the sweet spot — more workers don't help, and 32 MB chunks are actually slower than 4 MB due to allocation overhead and cache locality.flipFusesnow writes only the modified fuse wire bytes viahandle.write(..., position)instead of rewriting the entire file.getCurrentFuseWirestops scanning as soon as the first sentinel is found.Other changes
readFuseWire()helper that coalesces the header + wire read into a single syscall and removes duplication betweensetFuseWireandgetCurrentFuseWire.finallyblocks for proper cleanup on error.The public API is unchanged.